Arrow R-CNN for handwritten diagram recognition

Abstract We address the problem of offline handwritten diagram recognition. Recently, it has been shown that symbols can be directly recognized with deep learning object detectors. However, detectors are not able to recognize structure. propose Arrow R-CNN, first system for joint symbol and structure recognition in diagrams. R-CNN extends Faster detector an arrow head tail keypoint predictor a diagram-aware postprocessing method. network architecture data augmentation methods targeted at small datasets. Our method addresses insufficiencies standard postprocessing. It reconstructs from set detections keypoints. improves state-of-the-art substantially: on scanned flowchart dataset, we increase rate diagrams 37.7 78.6%.